In this paper, we introduce NeuRaLaTeX, which we believe to be the first deep learning library written entirely in LaTeX. As part of your LaTeX document you can specify the architecture of a neural network and its loss functions, define how to generate or load training data, and specify training hyperparameters and experiments. When the document is compiled, the LaTeX compiler will generate or load training data, train the network, run experiments, and generate figures. This paper generates a random 100 point spiral dataset, trains a two layer MLP on it, evaluates on a different random spiral dataset, produces plots and tables of results. The paper took 48 hours to compile and the entire source code for NeuRaLaTeX is contained within the source code of the paper. We propose two new metrics: the Written In Latex (WIL) metric measures the proportion of a machine learning library that is written in pure LaTeX, while the Source Code Of Method in Source Code of Paper (SCOMISCOP) metric measures the proportion of a paper's implementation that is contained within the paper source. We are state-of-the-art for both metrics, outperforming the ResNet and Transformer papers, as well as the PyTorch and Tensorflow libraries. Source code, documentation, videos, crypto scams and an invitation to invest in the commercialisation of NeuRaLaTeX are available at https://www.neuralatex.com

GPR full-waveform inversion optimizes the subsurface property model iteratively to match the entire waveform information. However, the model gradients derived from wavefield continuation often contain errors, such as ghost values and excessively large values at transmitter and receiver points. Furthermore, models updated based on these gradients frequently exhibit unclear characterization of anomalous bodies or false anomalies, making it challenging to obtain accurate inversion results. To address these issues, we introduced a novel full-waveform inversion (FWI) framework that incorporates an embedded convolutional neural network (CNN) to adaptively filter model parameters and gradients. Specifically, we embedded the CNN module before the forward modeling process and ensured the entire FWI process remains differentiable. This design leverages the auto-grad tool of the deep learning library, allowing model values to pass through the CNN module during forward computation and model gradients to pass through the CNN module during backpropagation. Experiments have shown that filtering the model parameters during forward computation and the model gradients during backpropagation can ultimately yield high-quality inversion results.

Software systems are increasingly relying on deep learning components, due to their remarkable capability of identifying complex data patterns and powering intelligent behaviour. A core enabler of this change in software development is the availability of easy-to-use deep learning libraries. Libraries like PyTorch and TensorFlow empower a large variety of intelligent systems, offering a multitude of algorithms and configuration options, applicable to numerous domains of systems. However, bugs in those popular deep learning libraries also may have dire consequences for the quality of systems they enable; thus, it is important to understand how bugs are identified and fixed in those libraries. Inspired by a study of Jia et al., which investigates the bug identification and fixing process at TensorFlow, we characterize bugs in the PyTorch library, a very popular deep learning framework. We investigate the causes and symptoms of bugs identified during PyTorch's development, and assess their locality within the project, and extract patterns of bug fixes. Our results highlight that PyTorch bugs are more like traditional software projects bugs, than related to deep learning characteristics. Finally, we also compare our results with the study on TensorFlow, highlighting similarities and differences across the bug identification and fixing process.

The Torch [36] have dramatically reduced the efforts of developers motivation of developing LibAUC is to address the convergence and researchers for implementing different DL methods without issues of existing libraries for solving these problems. In particular, worrying about low-level computations (e.g., automatic differentiation, existing libraries may not converge or require very large mini-batch tensor operations, etc). Based on these platforms, plenty sizes in order to attain good performance for these problems, due of DL libraries have been developed for different purposes, which to the usage of the standard mini-batch technique in the empirical can be organized into different categories including (i) supporting risk minimization (ERM) framework. Our library is for deep X-risk specific tasks [15, 35], e.g., TF-Ranking for LTR [35], VISSL for optimization (DXO) that has achieved great success in solving a variety self-supervised learning (SSL) [15], (ii) supporting specific data, of tasks for CID, LTR and CLR. The contributions of this paper e.g., DGL and DIG for graphs [31, 55]; (iii) supporting specific models include: (1) It introduces a new mini-batch based pipeline for implementing [13, 58, 59], e.g., Transformers for transformer models [59]. DXO algorithms, which differs from existing DL pipeline in However, it has been observed that these existing platforms and the design of controlled data samplers and dynamic mini-batch losses; libraries have encountered some unique challenges when solving (2) It provides extensive benchmarking experiments for ablation some classical and emerging problems in AI, including classification studies and comparison with existing libraries. The LibAUC library for imbalanced data (CID), learning to rank (LTR), contrastive features scalable performance for millions of items to be contrasted, learning of representations (CLR). In particular, prior works have faster and better convergence than existing libraries for optimizing observed that large mini-batch sizes are necessary to attain good X-risks, seamless PyTorch deployment and versatile APIs for various performance for these problems [4, 5, 7, 37, 43, 46], which restricts loss optimization. Our library is available to the open source the capabilities of these AI models in the real-world.

Providing online adaptive lightweight time series anomaly detection without human intervention and domain knowledge is highly valuable. Several such anomaly detection approaches have been introduced in the past years, but all of them were only implemented in one deep learning library. With the development of deep learning libraries, it is unclear how different deep learning libraries impact these anomaly detection approaches since there is no such evaluation available. Randomly choosing a deep learning library to implement an anomaly detection approach might not be able to show the true performance of the approach. It might also mislead users in believing one approach is better than another. Therefore, in this paper, we investigate the impact of deep learning libraries on online adaptive lightweight time series anomaly detection by implementing two state-of-the-art anomaly detection approaches in three well-known deep learning libraries and evaluating how these two approaches are individually affected by the three deep learning libraries. A series of experiments based on four real-world open-source time series datasets were conducted. The results provide a good reference to select an appropriate deep learning library for online adaptive lightweight anomaly detection.

AudioFlux is a Python library that provides deep learning tools for audio and music analysis and feature extraction. It supports various time-frequency analysis transformation methods, which are techniques for analyzing audio signals in both the time and frequency domains. Some examples of these transformation methods include the short-time Fourier transform (STFT), the constant-Q transform (CQT), and the wavelet transform. In addition to the time-frequency analysis transformations, AudioFlux also supports hundreds of corresponding time-domain and frequency-domain feature combinations. These features can be used to represent various characteristics of the audio signal, such as its spectral content, its temporal dynamics, and its rhythmic patterns.

Abstract: Object detection requires substantial labeling effort for learning robust models. Active learning can reduce this effort by intelligently selecting relevant examples to be annotated. However, selecting these examples properly without introducing a sampling bias with a negative impact on the generalization performance is not straightforward and most active learning techniques can not hold their promises on real-world benchmarks. In our evaluation paper, we focus on active learning techniques without a computational overhead besides inference, something we refer to as zero-cost active learning. In particular, we show that a key ingredient is not only the score on a bounding box level but also the technique used for aggregating the scores for ranking images.

Entropy, Cross-entropy, Binary Cross-entropy, and Categorical Cross-entropy are crucial concepts in Deep Learning and one of the main loss functions used to build Neural Networks. All of them derive from the same concept: Entropy, which may be familiar to you from physics and chemistry. However, not many courses or articles explain the terms in-depth, since it requires some time and mathematics to do it correctly. In the first post, I presented three different but related conceptions of entropy and where its formula derives from. However, there is still one key concept to address, since Deep Learning does not use Entropy but a close relative of it called Cross-entropy.

Recent advances in deep learning (dl) have led to the release of several dl software libraries such as pytorch, Caffe, and TensorFlow, in order to assist machine learning (ml) practitioners in developing and deploying state-of-the-art deep neural networks (DNN), but they are not able to properly cope with limitations in the dl libraries such as testing or data processing. In this paper, we present a qualitative and quantitative analysis of the most frequent dl libraries combination, the distribution of dl library dependencies across the ml workflow, and formulate a set of recommendations to (i) hardware builders for more optimized accelerators and (ii) library builder for more refined future releases. Our study is based on 1,484 open-source dl projects with 46,110 contributors selected based on their reputation. First, we found an increasing trend in the usage of deep learning libraries. Second, we highlight several usage patterns of deep learning libraries. In addition, we identify dependencies between dl libraries and the most frequent combination where we discover that pytorch and Scikit-learn and, Keras and TensorFlow are the most frequent combination in 18% and 14% of the projects. The developer uses two or three dl libraries in the same projects and tends to use different multiple dl libraries in both the same function and the same files. The developer shows patterns in using various deep-learning libraries and prefers simple functions with fewer arguments and straightforward goals. Finally, we present the implications of our findings for researchers, library maintainers, and hardware vendors.

A quick introduction to JAX, MXNet, MATLAB, and FluxPhoto by Gabriel Sollmann on UnsplashMachine learning libraries accelerate the deep learning revolution. They lowered the barrier of entry for practitioners by abstracting many difficult things such as GPU speedup, matrix algebra, and automatic differentiation. In both industries and academia, two deep learning libraries reign supreme: PyTorch and TensorFlow. In this article, I will introduce you to some other deep learning libraries that have considerable usage, either because they achieve speedup in some ways, or because they are used by very specific groups. Let’s begin!JAXhttps://medium.com/media/6bac18fd43fec2b80e2b35ffe642f650/hrefWhat is it? A open-source and in-development numerical framework originally developed by Google (think NumPy but for GPU).Who uses it? Many teams within Google, such as DeepMind.Why should you know about it? JAX was developed by Google to accelerate numerical computing on GPU and Google’s own hardware TPU. Using ideas such as accelerated linear algebra, just-in-time compilation (JIT), and automatic vectorization, JAX achieved great speedup and scale. Even though their syntax are similar to minimize the learning curve, JAX has a different design philosophy from NumPy. JAX encourages functional programming though functions such as vmap and pmap (vectorize + parallelize).Currently, many high-level APIs have been developed for JAX. Notable ones are Haiku and Flax.Apache MXNethttps://medium.com/media/ed0dbfe51985a97c4f9b1700242e23b7/hrefWhat is it? An open-source veteran machine learning framework with front-end bindings for multiple languages including Python, C++, R, Java, and Perl.Who uses it? Amazon AWS.Why should you know about it? MXNet most powerful features are its support for many programming languages and its scalability. Benchmark tests by NVIDIA shows that MXNet is faster than PyTorch and TensorFlow on some deep learning tasks.MXNet comes with Gluon, a high-level API to build neural networks. It also has an ecosystem for image classification (GluonCV) and NLP (GluonNLP).MATLAB Deep Learning Toolboxhttps://medium.com/media/f35c14825e5dd6f7974091faeb658ca1/hrefWhat is it? An add-on toolbox for MATLAB users that can create and train neural networks for a variety of tasks.Who uses it? Academia and industries such as aerospace and mechanical engineering. For example, Airbus used it to detect defects inside airplanes.Why should you know about it? Whatever you feel about MATLAB, it is still a popular programming ecosystem amongst academics and engineers. It has great user support and, in my opinion, the best documentation out of all the deep learning libraries in this list. The deep learning toolbox is geared toward people who want to build systems using minimal programming. Simulink, a graphical programming interface within MATLAB, offers ways to create easy-to-understand deep learning pipelines.Julia Fluxhttps://medium.com/media/c8849fbf650e0032d47f701a04573899/hrefWhat is it? An open-source machine learning library built for Julia programming language.Who uses it? Computing-intensive fields such as pharmaceuticals and finances. For example, Astrazeneca used it to predict drug toxicity.Why should you know about it? Julia programming language gained momentum over the years amongst data scientists, quants, and bioinformatics researchers. It is comparable to C/C++ in terms of speed, and it was designed to be beginners-friendly like Python. An implementation of Julia deep learning on Google TPU showed >200x speedup compared to CPU. If you are already coding in Julia, Flux is a great library to look into.ConclusionI hope that, with this short article, you are introduced to some other deep learning libraries. They all support efficient speedups, GPU scaling, and deployment into productions. There are excellent learning sources for all of them on the internet. Happy coding!Sources[1] https://www.deepmind.com/blog/using-jax-to-accelerate-our-research[2] https://github.com/aws/sagemaker-python-sdk[3] https://developer.nvidia.com/deep-learning-performance-training-inference[4] https://www.mathworks.com/company/user_stories/case-studies/airbus-uses-artificial-intelligence-and-deep-learning-for-automatic-defect-detection.html[5] https://twitter.com/jeffdean/status/1054951415339192321?lang=en[6] https://julialang.org/blog/2012/02/why-we-created-julia/[7] https://juliacomputing.com/case-studies/astra-zeneca/Not Just PyTorch and TensorFlow: 4 Other Deep Learning Libraries You Should Lnow was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.